Sprockets at Ten Years 


Key Findings from Ripple Effects Mapping 
Discussions among Sprockets Stakeholders 


Launched in 2011, Sprockets is a network of after-school and summer programs for youth in Saint Paul. 
A 2006 community-engaged process resulted in a vision for out-of-school time (OST) in Saint Paul that 
became Sprockets. That process involved the mayor’s office, Saint Paul Public Schools, community 
organizations and activists, local nonprofits, business and civic leaders, parents and youth, thought 
leaders, and public servants. Currently, Sprockets is a collaboration of community organizations, the 
city of Saint Paul, and Saint Paul Public Schools. It provides training, facilitation, and resources to OST 
programs and supports connections across organizations to provide high quality programming for Saint 
Paul youth and improve youth outcomes. 


In May 2021, Wilder Research worked with Sprockets to facilitate a series of three Ripple Effects 
Mapping (REM) sessions with Sprockets stakeholders to better understand the impacts of Sprockets in 
Saint Paul communities 10 years into its existence. 


About the approach 


REM is an evaluation method used to understand the intended and unintended impacts of complex 
community-based programs. It involves a facilitated discussion with program staff and local stakeholders 
and the creation of a visual “mind map” during the discussion that shows the links between program 
activities and changes or “ripples” in the community. 


Three REM discussions were held in May 2021. Sprockets worked with Wilder Research to identify 
potential attendees for the discussions. A total of 35 people attended the discussions (13 attended the 
first discussion, 5 attended the second, and 19 the third). Participants included representatives from OST 
programs, partner organizations, funders, and other stakeholders. Staff from Wilder Research facilitated 
the three REM discussions virtually through Zoom. 


This report summarizes the impacts of Sprockets identified during the discussions. During each of the 
discussions, participants worked with the facilitators to create a visual “mind map” of impacts. Wilder then 
combined information from all three maps to create an aggregate map using categories adapted from those 
identified during the discussions. 
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Ripple Effects Mapping 


This graphic shows the themes that emerged from three Ripple Effects Mapping discussions about the impact of Sprockets in Saint Paul and broader 


communities in its first 10 years. 


t} IMPACTING SYSTEMS 


Because of Sprockets, there is greater representation 
of youth voices in decision-making processes and an 
understanding that youth development is the 
responsibility of the whole community. Sprockets is also 
a national leader in the field of youth development in 
promoting authentic youth engagement, continuous 
quality improvement, and social and emotional learning. 


“It's really pushed some different models of how do we learn 
and where do we learn - not just in a school building.” 


“It's normalized the idea that youth should be at the center of 
decision-making. It's not a joke anymore to consider youth as 
the driver, and de-centering adults from everything.” 


SUPPORTING STRATEGIC COLLABORATION TO 
BETTER MEET THE NEEDS OF YOUTH 


Sprockets has created a network of OST programs 
supporting greater collaboration, learning, and 
field-building. Sprockets has also fostered relationships 
between OST programs and funders, decision-makers, 
and other stakeholders. 


“The network has contributed to more of a positive sum 
framework - it's less that we have to compete, but more of a 
"we all do better when we all do better" approach by being an 
active network.” 


HELPING OST PROGRAMS USE DATA TO IMPROVE 
AND SCALE PROGRAMS 


Sprockets created a citywide system to share and track 
data and helps OST programs of all sizes use data to 
improve and scale their programs. 


“We are a very small organization. Because of the statistical 
data, we were able to get a grant to scale up the program.” 


“The way that Sprockets has brought evaluative thinking to the 
people in an accessible way, it removes the angst, but also sets 
the standard for quality work.” 
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CREATING A SUSTAINABLE WAY TO SHARE BEST 
PRACTICES AND SUPPORT PROFESSIONAL DEVELOPMENT 


Sprockets provides accessible trainings and resources to 
support continuous quality improvement and professional 
development for youth workers. 


“Something that Sprockets does really well is take something 
that's growing in the field and bring it to Saint Paul in a meaningful 
way, but also contextualize it.” 


“I've attended all the possible trainings, and it's made a difference 
in how | approach working with the children.” 


“For us as a funder that doesn't have as deep of an expertise ... 
the connection to Sprockets has helped us better gauge program 
quality, tap into their expertise.” 


FOSTERING AUTHENTIC ENGAGEMENT AND LEADERSHIP 
OPPORTUNITIES FOR YOUTH 


Sprockets models authentic youth engagement by 
providing opportunities for youth to contribute their voices, 
participate in decision-making processes, and serve as 
leaders. 


“With their community engagement team, Sprockets took to heart 
how they engage young people. They've been bringing in youth to 
help with Listen Up, to help young people get data from their peers 
and then present that to stakeholders.” 


SUPPORTING PROGRAMS’ CAPACITY TO REACH YOUTH 


Through initiatives like its online Program Finder, 
Sprockets has made it easier for youth and families to 
become connected to OST programs. Sprockets has also 
provided support to OST programs to better reach youth 
and families. 


“For parents to go on the website [and search for programs by] 
where they're located, what they're looking for... it's also helpful 
for us as organizations. ... When you make things easy for parents 
or our community, we are really serving them and offering them 
the tools they need.” 


Project impacts 


Supporting strategic collaboration to better meet the needs 
of youth 


Stakeholders mentioned how Sprockets has intentionally fostered collaboration and a community of 
practice among OST programs and partners. This network provided an avenue for programs to share 
promising practices, connect or reconnect with other organizations, and coordinate programs to better 
meet the needs of youth. The network also allowed youth workers to stay connected to each other even 
as partnerships shifted or as they moved to other organizations. Several participants mentioned that 
connections they made through Sprockets led to collaboration on other grants or programs. A number of 
participants emphasized the intentionality of Sprockets’ approach to building a network of OST programs 
as a contributing factor to its success. 


Sprockets has removed our excuse to stay siloed, made it so easy to learn about new organizations and 
share learning spaces. We have no excuse to do our own work and not collaborate. 


The collaboration and knowing there are people in the network who do things better than we do, we can 
reach out to them. 


[Sprockets] helps youth workers stay connected to one another even if they're moving around to other 
organizations. It also helps folks stay connected to organizations even if the staff are shifting a lot. There's a 
lot of connection that Sprockets facilitates that individuals couldn't sustain. Sprockets is the connection 
point, and it's a highly mobile field, so it's a huge value. 


Just inviting people into a network or community of practice. That's not just something that happens, it was 
intentionally created and continues to be supported. Identifying where the programs were and linking 
information to see where there are gaps. That's a lot of work to keep up, but at the end of the day, it's not 
just about information but where are we missing young people and how can we step up to support them. 


Participants also mentioned how Sprockets has created spaces for relationships to be built between 
funders and OST programs as well as with other stakeholders in the region, such as advocacy 
organizations, policymakers, and OST programs outside of Saint Paul. 


As a funder, being a part of Sprockets immersed me into what this youth development space looks like in 
Saint Paul, and it's also helped me make connections to nonprofits that are struggling to effectively do youth 
programming. It's always been a really valuable resource for relationship-building, especially in those 
meeting spaces. 


There is more regional leadership and collaboration outside of Saint Paul. 
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Supporting programs' capacity to reach youth 


Participants described a number of ways Sprockets had helped individual programs, and OST programs 
in general, reach more youth in Saint Paul. Several pointed to Sprockets’ Program Finder, an online 
search engine where youth and families can find OST programs that meet their needs. One participant 
reflected that because of the infrastructure provided by Sprockets, there is less turnover in OST programs, 
which provides more stability for youth. Another participant said that the support Sprocket provides, 
especially related to professional development and evaluation, frees up resources that can then be used 
for reaching youth. Likewise, the network building and evaluation work that Sprockets does has increased 
understanding of barriers to OST participation more broadly across the region, which programs could 
use to inform their outreach efforts. 


Ten years ago, | couldn't have told you what the programs are around Saint Paul ... we shouldn't gloss over 
the Program Finder as a major outcome.... Before it was who you knew, so it wasn't equitable. Young 
people who weren't connected to those programs didn't have as much access. 


My sense as someone who's done youth work, there's less coming and going of small youth programs. It 
gives more stability for small youth programs to continue.... A lot of the programs that popped up in areas 
like Frogtown were situated in that specific context and didn't always make it, but should have. 


The tools and services they offer to organizations at low or no cost, those can be put towards other 
programs to increase access or trainings, that can impact our ability to address inequities. 


Fostering authentic engagement and leadership opportunities 
for youth 


Participants described ways in which Sprockets modelled authentic engagement of young people, 
providing opportunities for youth to contribute their voices, participate in decision-making processes, 
and serve as leaders. Examples included: a mayoral candidate forum moderated by youth; Youth on 
Board, a program connecting youth to positions on boards and commissions; and the Listen Up 
initiative, where Sprockets hired 10 youth to do community engagement and gather survey data from 
their peers about OST programs. 


Sprockets does a really good job of walking the walk, they also employ young people in addition to the 
systems level work. That's hard to do — both systems things and applying it day-to-day in partnering and 
coordination with young people. 


| think of the group of young people who went around and gathered input from other young people in a wide 
variety of communities to make sure they captured perspectives and voices. To have young people see that 
information they gathered ... and presented to the school board and mayor, they were able to say here's 
what we did, here's what we found, here's what we need. They know their recommendations are being 
considered. It does change - adults don't always know what's best for young people. I've been impressed 
with the genuine opportunities that young people have been able to have. 
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Creating a sustainable way to share best practices and support 
professional development 


Training and professional development were among the key benefits participants cited for being part of 
the Sprockets network. This was especially helpful for new staff getting oriented to the field of youth 
development, and suggested that in addition to the skills and relationships they built, the trainings also 
offered a sense of professional identity and career path. Funders also mentioned that the professional 
development events helped them learn about trends in the field and better assess program quality. A 
number of participants mentioned specific changes they made to their programs as a result of the tools 
offered by Sprockets, including a more intentional focus on social-emotional learning (SEL) and use of 
the Youth Program Quality Assessment (YPQA). 


Participants appreciated that Sprockets brought in tools and promising practices from the national level, 
contextualized for the programs in Saint Paul. Several participants suggested that Sprockets did not just 
offer individual trainings, but a cohesive set of tools and resources and an infrastructure for supporting the 
professional development of youth workers. These participants appreciated that Sprockets made trainings 
accessible for organizations of all sizes and resources, and a representative from another city mentioned 
that they look to Sprockets as a model and resource for trends and promising practices in the field. 


Oftentimes in smaller organizations, youth workers are pretty siloed and may not be aware that there's 
others out there with a shared set of values, and that discovery that this is a profession helps elevate the 
quality of the work, the experience of the people doing the work, and the experiences of the young people. 


We at [a funding organization] can use Sprockets to get a sense of what's happening in the field without 
burdening our grantees. It's a more equal relationship so we get a better sense of what's happening. We 
use Sprockets to get a better sense of community needs. We did a pilot about transportation needs, and it 
was helpful to have Sprockets as a partner. 


For [our organization], before that partnership, we had not given thought to YPQA and SEL especially in 
regards to our out-of-school programming. It's now part and partial to the lens we use to look at OST programs. 


Helping OST programs use data to improve and scale programs 


Since 2011, Sprockets has worked with Wilder Research to manage the Sprockets shared-data system to 
track youth participation in OST programs, gathering data from OST providers and linking the data to 
Saint Paul Public Schools student data. Sprockets has also provided trainings and professional development 
in evaluation and quality improvement, including support for using the YPQA and other assessment 
tools. A number of representatives from OST programs noted how the evaluation support, especially the 
shared-data system, allowed them to better make the case to funders for investing in their programs. This 
was especially helpful for smaller programs, and several representatives from these programs pointed to 
ways it allowed them to scale up their programs. Other participants described how quality improvement 
practices and evaluative thinking are now a standard part of their work because of the support provided 
by Sprockets. The shared-data system also allowed programs to get aggregate OST data across programs. 
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Several funders observed an overall increase in the quality of programs through Sprockets. Because of 
Sprockets’ investment in evaluation and quality improvement, programs were better able to articulate 
their theories of change and impact, and funders could more easily identify promising programs. 


| come from a different perspective as a director of a small program that had an opportunity from Sprockets 
to be a bigger drop in the water. Knowing our kids were registered in the Sprockets database, and knowing 
our involvement with kids was being recognized. 


With the evaluation thinking model, it's formative and supportive, helping groups and organizations coming 
in at where they are now and take that next step to look at their data. 


In a former role | had as a funder, | noticed programs that were part of Sprockets were able to clearly 
articulate their program intention, design, and evaluation. In the applications they submitted, | could tell they 
were part of Sprockets. 


The research side says that when you're doing continuous quality improvement, as a funder there's more 
integrity because you can see what's not working so you can improve it. Providing professional development 
as that intermediary — | can't think of high-risk grantees in Saint Paul because of Sprockets where we'd need 
to invest more in monitoring and professional development. 


Impacting systems 


Participants discussed how Sprockets not only impacted youth programs and families, but was also 
influencing systems of power. In particular, they noted a greater recognition among policymakers and 
organizations that youth development was the responsibility of all members of the community and did 
not just occur in school. They had also seen policymakers and organizations center youth and family 
voices more intentionally in decision-making processes, including having youth sit on boards and 
commissions. Sprockets also created a citywide infrastructure for doing youth work, including a shared 
data system. This infrastructure also increased brand recognition of OST programs and youth development 
as concepts among families, organizations, and policymakers. Some participants also indicated how 
Sprockets was influencing the field of youth development nationally, including being a leader on 
authentic youth engagement, quality improvement, and social-emotional learning. 


There's more of a recognition that it's part of the work, not just of the school system to deal with youth. It's part 
of the work of all of city government. It's part of the work of multiple organizations. It's not just one groups' 
responsibility. It puts the responsibility for bringing our young people up on everyone. 


The idea of centering youth voice has become a more significant part with the school district and city. 
Sprockets offered a way, this is how you do it, it's not just imaginary to center youth voice. You can have 
effective programs that run this way. Youth councils, the Saint Paul Public Schools Student Engagement 
Advisory Board. It's not just this is nice to have young people at the table. Decisions can be made better with 
young people at the table. 


The city before Sprockets, all the neighborhoods and organizations operated solely, and in competition often. 
You didn't go from the Westside to Eastside to learn how to do something. You didn't share across neighborhoods. 
Now people do that. You don't say | don't know those people so | can't talk to them. So there's been much 
more citywide work, which has been a strength to the whole city. 


| always wish others would understand how special it is for Saint Paul to have Sprockets and how it's 
nationally recognized and how it's part of important conversations about the direction of the field. 
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Future directions 


During the REM discussions, participants were also asked to reflect on future directions for Sprockets 
work. The following themes reflect what participants shared: 


m Serving as a convener. Several respondents suggested that Sprockets could serve as a more 
proactive convener of organizations working in similar areas, such as sports, or among funders that 
support youth development programs. One participant commented that Sprockets could play a 
coordinating role around space usage, especially among schools, churches, or community centers 
that have often have unused space. 


= Engaging communities. Although participants recognized centering youth voice as a strength of 
Sprockets, some commented that Sprockets could further engage youth and families or offer greater 
language accessibility for resources like the Program Finder. 


= Exploring new research opportunities. Participants were interested in new ways to use research to 
document Sprockets’ impact or better understand community needs. Potential research projects 
included a comparison study of Saint Paul versus a community without an OST network like 
Sprockets, mapping OST programs to assess Sprockets’ impact and gaps in services, and examining 
how Sprockets mobilized during the COVID-19 pandemic. One participant suggested creating an 
app that could be used by youth and families to enter and track their own data. Another respondent 
recommended engaging community members in looking at the data Sprockets gathers. 


= Building strategic partnerships. A few participants recommended that Sprockets pursue or deepen 
specific partnerships, including the Saint Paul Public Schools or Ramsey County to expand the 
program countywide. One participant wondered about whether relationship-building was needed 
with the current Saint Paul mayor and Saint Paul Public Schools superintendent who both were not 
involved at the beginning of Sprockets. 


m Advancing equity. Participants encouraged Sprockets to leverage its power and reputation to have a 
stronger voice in advancing racial equity. Some questioned whether Sprockets could use its network 
to mobilize for social change or have a stronger advocacy role at the policy level. 


= Rethinking what youth development looks like after the pandemic. Several participants expressed 
disappointment that the city had not engaged OST programs more in discussions about how to 
support young people during the pandemic. Some reflected that Sprockets could build on its work 
before the pandemic to create learning campuses, along with lessons learned from schools and other 
youth-serving programs during the pandemic, to figure out what education and youth development 
could look like moving forward. 


m= Securing more sustainable funding. Some participants felt that Sprockets could play a more active 
role in advocating for more funding for OST programs, especially at the city or state levels. Another 
participant commented that Sprockets could serve as an administrator for federal funding for OST 
programs rather than having individual programs apply for funding. Another individual emphasized 
the importance of ensuring that Sprockets has sustainable funding into the future. 
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Appendix A: The Ripple Effects Mapping Method 


REM is an evaluation tool that was developed to document the intended and unintended impacts of 
complex community-based initiatives that are not easily captured by other methods of evaluation. The 
method involves a facilitated discussion with project staff, partners, community members, and other 
local stakeholders and the creation a visual “mind map” during the discussion, which shows links 
between program activities and resulting changes in the community. 


REM discussions typically involve 12 to 20 stakeholders representing a wide variety of roles and 
perspectives. After introductions, the two-hour discussion begins with participants splitting into pairs to 
discuss what they found to be the most important change that has come out of the project. These ideas 
are shared with the large group and mapped onto a projected screen using a mind-mapping software. For 
this project, X-Mind software was used to create the maps. The facilitator then leads a discussion to 
organize the responses into themes and explore the “ripples” of these impacts by asking questions such 
as “What happened as a result of this?” or “What led to this?” By the end of the session, a draft of the 
map is completed that incorporates the feedback from all of the participants. 


For this report, the data were organized using a coding scheme developed by Wilder Research that 
reflected themes that emerged during the discussions. This approach helps ensure that the local 
knowledge was captured and presented as the group understood it. 


Wi Ider For more information about this report, contact Edith Gozali-Lee at 
edith.gozali-lee@wilder.org 
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